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Evaluation  of  a  New  Method  for  Assessing  Change  to  Planned  Job  Redesign 
as  Applied  to  Hackman  and  Oldham's  Job  Characteristic  Model 

Implicit  in  the  job  design  literature  is  the  notion  that  properly 
planned  changes  in  job  activities  will  result  in  improved  employee  atti¬ 
tudes  and  behaviors  (cf,  Herzberg,  1966;  Scott,  1966;  Emory  and  Trist, 

1969;  Hackman  and  Oldham,  1976;  Keller,  Szilagyi  and  Holland,  1976).  The 
empirical  literature,  however,  only  infrequently  includes  reports  of  em¬ 
ployee  reactions  to  job  characteristics  both  before  and  after  job  redesign 
has  taken  place.  Furthermore,  when  such  research  designs  are  used  the 
issue  of  how  to  unambiguously  measure  change  can  become  a  problem. 

The  goal  of  this  research  is  to  consider  the  utility  of  a  new  approach 
to  the  measurement  of  change  as  applied  to  the  Hackman  and  Oldham  model 
(1976).  This  approach  draws  heavily  from  the  work  of  Howard  (Howard,  Ralph, 
Gulanick,  Maxwell,  Nance  and  Gerber,  1979;  Howard  and  Dailey,  1979;  Howard, 
Schmeck  and  Bray,  1979)  on  response  shift  bias.  The  potential  contribution 
of  this  research  lies  in  the  ability  of  response  shift  methodology  to  pro¬ 
vide  a  more  valid  test  of  employee  reactions  to  job  redesign. 

Critical  Aspects  of  the  Hackman  and  Oldham  Model  of  Job  Design 

The  job  characteristics  model  proposed  by  Hackman  and  Oldham  focuses  on 
five  core  job  dimensions  that  are  hypothesized  to  be  related  to  key  psycho¬ 
logical  states  and  to  various  personal  and  work  outcomes.  These  dimensions 
are:  skill  variety,  task  identity,  task  significance,  autonomy  and  feedback. 

The  Job  Diagnostic  Survey  (JDS)  was  developed  to  measure  each  of  these 
core  dimensions  (Hackman  and  Oldham,  1975).  Using  the  JDS,  it  is  possible 
to  compute  a  summary  score  indicating  the  overall  motivating  potential  score. 
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or  MPS ,  of  a  job.  Individuals  who  describe  their  jobs  with  high  MI’S  scores 
are  predicted  to  display  high  internal  motivation,  high  quality  of  work,  low 
absenteeism,  and  low  turnover,  and  to  report  high  job  satisfaction. 

Within  the  confines  of  the  model,  it  is  important  to  recognize  two 
critical  and  seemingly  obvious  aspects  that  are  rarely  considered  jointly 
when  the  model  is  tested.  The  first  aspect  is  the  dynamic  nature  of  the 
model.  It  is  based  on  the  idea  of  change .  One  of  the  main  reasons  that 
Hackman  and  Oldham  developed  the  model  was  to  have  it  serve  as  a  framework 
for  changing  jobs  (Roberts  and  Glick,  in  press).  Embedded  in  this  notion 
of  change  is  the  second  critical  aspect.  Incumbent  percept  ions  of  the  job 
are  the  appropriate  criteria  for  determining  whether  or  not  the  job  has 
changed.  Although  Hackman  and  Oldham  (1975)  originally  recommend  that  .IDS 
ratings  be  supplemented  by  independent  assessments  made  by  individuals  who 
are  not  incumbents  of  the  focal  job,  they  also  admit  that  "when  the  intent 
is  to  predict  or  understand  employee  attitudes  or  behavior  at  work,  employee 
ratings  of  the  job  dimension  should  be  used  since  it  is  the  employees'  own 
perception  of  the  objective  job  that  is  causal  of  his  reactions  to  it" 

(Hackman  and  Oldham,  1975,  p.  169). 

In  summary,  the  job  characteristics  model  proposed  by  Hackman  and  Oldham 
is  a  within-subj ect s  model  that  is  concerned  with  an  individual's  perceptions 
and  reactions  to  a  situation  and  how  these  perceptions  impact  on  attitudes  and 
behavior.  Changes  in  the  nature  of  the  job  are  thought  to  produce  different 
perceptions  of  the  job  and  this  leads  to  changes  in  attitudes  and  behaviors. 
The  critical  link  in  this  causal  model  seems  to  be  perceptions  of  differences 
in  the  job  by  the  incumbent  stemming  from  objective  changes  due  to  an  inter¬ 
vention.  The  greater  the  perceived  change  an  individual  "sees"  the  greater 
the  change  In  that  individual's  attitudes  and  behaviors. 


Eva luat  imi  of  i  hangc 
4 


Review  of  Literature 

Examination  of  past  research  reveals  that  the  various  designs  most  often 
used  to  test  the  Hackman  and  Oldham  model  usually  Ignore  one  or  t  lie  other  u! 
the  two  aspects  of  the  theory  just  discussed,  and  consequently  these  designs 
provide  differential  support. 

One  of  the  more  frequent  designs  used  is  correlational  in  nature.  The 
JDS  is  administered  to  a  group  of  employees  and  either  core  dimension  scores 
or  MPS  scores  are  correlated  with  outcomes  such  as  performance,  absenteeism, 
turnover,  satisfaction,  and  internal  work  motivation.  The  job  is  not  clianged 
and  incumbents'  perceptions  of  the  job  and  reactions  to  the  job  are  measured 
at  one  point  in  time. 

Several  studies  were  found  that  used  this  correlat iona 1  design  (cf.  Brief 
and  Aldag,  1975;  Evans,  Kiggundu  and  House,  1979;  Hackman  and  Oldham,  1975, 

1976;  Keller,  Szilagyi  and  Holland,  1976;  and  Stone  and  Porter,  1975).  Taken 
as  a  whole  these  correlational  studies  provide  positive  albeit  limited  support 
for  the  model. 

The  inadequacy  of  the  correlational  design  to  test  the  Hackman  and  Oldham 
model  logically  leads  to  the  use  of  experimental  and  qu  isi-exper iment.i  1  designs.  In 
this  approach,  the  job  is  changed,  either  by  a  planned  intervention  or  due  to 
some  naturally  occurring  event.  Typically,  these  studies  employ  what  Campbell 
and  Stanley  (1963)  would  call  a  nonequivalent  control  group  design  in  which 
Pre-test  and  Post-test  are  used  to  collect  self-report  measures  of  the  five 
core  dimensions  and  employee  attitudes  for  both  the  treatment  group  and  con¬ 
trol  group. 

A  review  of  the  literature  produced  four  experimental  tests  of  the  Hackman 
and  Oldham  model  (cf.  Frank  and  Hackman,  1975;  Hackman,  Oldham.  Jansen  and 
Purdy,  1975;  Hackman,  Pearce  and  Wolfe,  1978;  and  Umstot ,  Bell  and  Mitchell, 
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1  9  7  b)  .  The  overall  findings  were  inconsistent  and  generally  less  supportive 
than  the  correlational  studies. 

One  explanation  for  different  results  produced  hv  correlational  methods 
and  experimental  methods  might  be  tlvat  In  tbe  experimental  studies  the  job 
did  not  markedly  change.  This  would  produce  the  pattern  of  more  negative 
results.  But,  this  explanation  seems  to  apply  primarily  to  Frank  and  Hai kman 
H475)  . 

A  second  explanation  stems  from  the  use  of  incumbents'  perceptions  to 
measure  not  only  the  degree  to  which  the  objective  job  characteristic*  had 
changed,  but  also  their  affective  reactions  to  the  job  changes.  The  experi¬ 
mental  studies  implicitly  test  the  notion  that  changes  in  job  characteristics 
change  incumbent  perceptions  of  the  job  which  in  turn  change  attitudes  and 
behaviors  on  the  job.  These  studies,  however,  assume  that  changes  in  job 
characteristics  are  the  primary  thing  that  affect  perceptions  of  the  job. 

But,  Salancik  and  Pfeffer  (1977)  have  argued  that  social  cues  might  also  in¬ 
fluence  job  perceptions.  White  and  Mitchell  (1979)  and  O'Reilly  and  Caldwell 
(1979)  have  empirically  demonstrated  this  possibility.  Priming  is  another 
factor  mentioned  by  Salancik  and  Pfeffer  (1977).  Priming  refers  to  the  possi¬ 
bility  that  obtrusive  data  collection  might  make  various  aspects  of  the  situa¬ 
tion  more  salient  than  they  might  b«  otherwise.  If  this  happens,  it  should 
produce  artificially  inflated  correlations  between  job  characteristics  and 
attitudes,  which  it  may  well  do  in  the  correlational  studies.  Using  Salancik 
and  Pfeffer's  (1977)  logic,  however,  priming  should  have  the  same  effect  on 
the  experimental  studies,  which  it  does  not  always  do  (see  Frank  and  Hackman, 
1975;  Hackman,  Pearce  and  Wolfe,  1978;  and  O'Reilly  and  Caldwell,  1979  as  ex¬ 
amples),  and  thus  does  not  explain  the  difference  between  the  results  produced 
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A  third  factor  that  may  affect  perceptions  of  the  job  is  the  frame  of 
reference  of  the  perceiver.  Past  experiences,  perceptions  of  available  al¬ 
ternatives  and  expectations  provide  anchors  that  are  used  to  scale  a  given 
phenomenon  (Smith,  Kendall  and  Hulin,  1969).  This  suggests  that  differences 
in  perception  by  employees  on  the  same  job  may  be  due  in  part  to  different 
frames  of  reference.  The  Job  Characteristics  Model,  however,  is  a  within- 
subjects  model  so  in  one  sense  inter-rater  agreement  is  not  important.  But, 
something  that  would  be  important  for  tests  of  a  within-subj ects  model  is 
intra-rater  agreement.  An  employee's  Pre-ratings  may  be  based  on  one  set  of 
anchor  points  which  in  turn  are  based  on  the  employee's  past  experience  with 
the  job  and  other  jobs.  Post-ratings  of  the  job  characteristics  after  job  re¬ 
design,  however,  may  be  based  on  a  different  set  of  anchors  because  receiving 
treatment,  i.e.,  job  enrichment,  changes  the  employee's  evaluation  standard 
with  regard  to  the  dimension  being  measured.  For  example,  an  employee  might 
believe  that  a  substantial  amount  of  autonomy  exists  in  the  job  and  as  a  re¬ 
sult,  this  dimension  is  given  a  score  of  seven  on  the  JDS  scale  at  Pre-test. 
But,  after  the  job  has  been  enriched,  the  same  employee  realizes  the  full  range 
in  amount  of  autonomy  a  job  can  have  and  rates  the  new  job  (which  actually 
allows  more  freedom  in  scheduling  and  doing  the  work)  as  a  six  on  the  JDS  au¬ 
tonomy  scale.  Furthermore,  the  employee  may  tell  you  that  the  enriched  job 
has  more  autonomy  and  that  the  Pre-test  score  was  inaccurate. 

Response  Shift  Bias 

A  change  in  the  frame  of  reference  of  the  perceiver  resulting  from 
planned  intervention  is  what  Howard  calls  a  "response  shift."  A  response  shift 
is  similar  to  Beta  change  (cf.  Golembiewski ,  Billingsley  and  Yeager,  1976; 
I'erborg,  Howard  and  Maxwell,  1980). 
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In  a  series  of  12  studies  Howard  and  his  colleagues  (Howard,  Ralph, 
Gulanick,  Maxwell,  Nance  and  Gerber,  1979;  Howard  and  Daily,  1979;  Howard, 
Schraeck  and  Bray,  1979;  Howard,  Daily  and  Gulanick,  in  press;  Howard,  Mill  hum, 
Slaten  and  O'Donnell,  in  press;  and  Bray  and  Howard,  in  press)  demonstrated 
that  self  report  ratings  at  Pre-test  may  be  based  on  a  different  frame  of 
reference  than  self-report  ratings  at  Post-test.  Thus,  comparison  of  Pro 
and  Post  intervention  scores  may  not  accurately  reflect  perceived  change  as 
conceptualized  by  the  person  at  Post-test. 

To  assure  that  before  and  after  ratings  are  made  on  the  same  subjective 
scale,  Howard  et  al  (1979)  recommend  that  at  Post-test,  each  person  be  asked 
to  make  two  ratings.  The  first  rating  is  the  usual  Post-test  description  of 
the  phenomenon  under  investigation  after  the  planned  intervention.  The  second 
rating,  however,  asks  the  person  to  describe  the  phenomenon  as  it  is  now  _per_- 
ceived  to  have  been  just  before  the  intervention.  This  retrospective  rating 
is  called  the  "Then"  measure  by  Howard.  If  Pre-ratings  and  Then  ratings  are 
different,  a  response  shift  is  said  to  have  occurred.  This  suggests  that  per¬ 
ceived  change  resulting  from  the  invervention  should  not  be  assessed  as  Post- 
Pre  scores.  Rather,  Howard  et  al  (1979)  recommend  that  Post-Then  scores  be 
used.  It  should  be  noted  that  Howard  et  al  (1979)  do  not  predict  the  occur¬ 
rence  of  response  shifts  to  all  planned  interventions.  Rather,  they  offer  a 
method  for  detecting  response  shifts  and  for  measuring  change  when  response 
shifts  are  found. 

The  implication  of  a  response  shift  for  tests  of  the  Job  Characteristics 
Model  is  straightforward.  Returning  to  the  earlier  example,  a  person  may  rate 
autonomy  as  a  seven  at  Pre-test,  a  six  at  Post-test,  and  a  three  using  the  Then 
rating.  Past  experimental  tests  of  job  redesign  models  have  neglected  to  con¬ 
sider  the  possibility  of  a  response  shift.  Consequently,  our  hypothetical  Pre 
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and  Post  scores  would  indicate  that  the  person  perceived  a  slight  decrease 
in  autonomy  whereas  the  Post  and  Then  scores  would  indicate  a  substantial 
perceived  increase.  Existence  of  a  response  shift  could  explain  the  nega¬ 
tive  results  found  by  Frank  and  Hackman  (1975),  the  failure  of  several  core 
dimensions  to  show  change  after  enrichment  in  the  IJmstot  et  al  study  (1976), 
and  the  weak  effects  found  for  job  characteristics  reported  by  O'Reilly  and 
Caldwell  (1979)  and  White  and  Mitchell  (1979).  Response  shift  effects  also 
could  contribute  to  the  observation  of  more  positive  results  found  with  cor¬ 
relational  studies  as  compared  with  experimental  studies.  Because  in  cor¬ 
relational  designs  all  the  data  are  collected  at  the  same  time,  changes  in 
frames  of  reference  are  unlikely. 

Before  we  totally  endorse  the  response  shift  methodology,  the  technique 
must  demonstrate  validity.  Howard  has  investigated  this  question.  Howard, 
Schmeck  and  Bray  (1979)  and  Howard  and  Daily  (1979)  found  that  different  con¬ 
clusions  about  the  effectiveness  of  an  intervention  could  be  made  depending  on 
whether  Post-Pre  scores  or  Post-Then  scores  were  used.  Howard  et  al  (1979), 
Howard  and  Daily  (1979),  and  Howard,  Schmeck  and  Bray  (1979)  found  that  Post- 
Then  scores  as  compared  with  Post-Pre  scores  were  more  highly  correlated  with 
independent  and  objective  assessments  of  change,  and  consequently  were  more 
valid.  Several  other  studies  also  demonstrated  that  Pre-Then  differences  were 
not  the  result  of  memory  biases  (Howard  et  al,  1979)  or  social  desirability 
(Howard,  Millham,  Slaten  and  O’Donnell,  in  press). 

The  Research  Problem 

The  present  research  is  an  experimental  study  of  the  Hackman  .mt!  Oldli.un  rmn 


that  will  attempt  to  extend  the  research  on  response  shift  bias  to  the  evalua¬ 
tion  of  job  redesign  programs  in  order  to  ascertain  if  the  existing  exporimen- 
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tal  and  longitudinal  studies  of  job  enrichment  might  be  compromised  bv 
response  shifts.  Two  major  research  questions  are  addressed: 

1)  Can  the  response  shift  be  shown  to  exist  in  ratings  of  job  cliarar- 
ter ist ics? 

2)  If  so,  does  the  retrospective  rating  provide  a  more  valid  predictor 
of  individual  change? 

In  order  to  answer  these  questions,  however,  other  conditions  must  be 
satisfied.  When  considering  the  utility  of  a  new  approach  to  the  measurement 
of  change,  it  is  important  that  the  new  approach  be  methodologically  sound. 

The  use  of  Then  ratings  should  not  result  in  methodological  artifacts.  Also, 
some  support  for  the  Hackman  and  Oldham  model  should  be  found.  The  validity 
of  retrospective  Then  ratings  cannot  be  demonstrated  if  the  Job  Characteristics 
Model  is  incorrect  or  the  manipulations  did  not  work. 

In  order  to  check  on  these  methodological  concerns,  the  present  study  was 
designed  to  investigate  the  following  methodological  questions  in  addition  to 
the  main  research  questions: 

1)  Are  different  experimental  groups  working  on  the  same  task  equivalent 
at  Pre-test? 

2)  Does  administration  of  a  Pre-test  measure  affect  either  the  Post-test 
or  Then  measure? 

3)  Does  administration  of  a  Post-test  measure  affect  the  Then  rating? 

A)  Does  administration  of  the  Then  measure  affect  the  Post-test  measure? 

5)  Does  knowledge  of  Post-test  responses  affect  the  Then  responses  and 
vice  versa?  In  other  words,  is  there  an  order  effect  when  presenting 
the  Post  and  Then  measures? 

6)  Did  the  manipulations  have  an  effect  using  the  conventional  Pre  and 


Post  measures? 
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METHOD 


Overview 


Subjects  worked  a  total  of  five  hours  in  two  sessions  on  a  job  that  they 
thought  was  for  a  proposed  student-work  program.  A  similar  situation  was  used 
by  O'Reilly  and  Caldwell  (1979).  The  task  consisted  of  doing  various  activi¬ 
ties  associated  with  coding  and  evaluating  videotaped  personnel  selection  in¬ 
terviews.  Two  interview  evaluation  tasks,  one  enriched  and  one  unenriched, 
were  designed.  In  order  to  answer  the  proposed  methodological  and  research 
questions,  five  groups  of  subjects  were  formed.  Although  subjects  viewed  the 
videotapes  in  small  groups,  they  worked  independently. 

Sj ibjacts  and  Assignment  to  Treatment  Groups 

Fifty-one  advanced  undergraduate  students  recruited  from  courses  in  in¬ 
dustrial  psychology  served  as  subjects.  One  subject  was  dropped  from  analysis 
because  a  fire  destroyed  her  apartment  and  she  was  unable  to  attend  the  second 
session  of  the  experiment.  Two  additional  subjects  were  dropped  hecuu  a 
flood  prevented  them  from  attending  the  second  work  session.  No  subjeits  wore 
dropped  because  of  hurricanes  or  locust.  All  subjects  had  received  prior  t ra 1 
ing  on  perse nnel  interviewing  in  their  psychology  classes.  Average  ago  was 
17.5  years  and  4 4%  of  the  subjects  were  male. 

Students  were  contacted  in  class  and  asked  to  help  in  evaluating  data  col 
looted  by  the  Interviewing  Institute  at  the  University  of  Houston.  Extra 
credit  was  offered  as  an  incentive.  Students  were  told  that  various  act  ivitio 
wore  being  considered  for  a  student  work-study  job  and  that  their  exposure  to 
interviewing  would  be  helpful  in  evaluating  the  feasibility  ot  treat  Ing  i Im¬ 
position.  Subjects  were  not  randomly  assigned  to  treatment  groups,  rather. 
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each  treatment  group  consisted  of  the  subjects  who  volunteered  from  a  par¬ 
ticular  class.  But,  assignment  of  a  particular  treatment  condition  to  a 
group  was  random.  Intact  groups  were  used  for  two  reasons:  1)  it  was 
feared  that  mixing  subjects  across  classes  would  increase  the  threats  of 
diffusion  or  imitation  of  the  experimental  treatment  and  hypothesis  guessing 
within  experimental  conditions  (Cook  and  Campbell,  1976)  and  2)  because  an 
advanced  understanding  of  interviewing  skills  was  required,  subjects  with  the 
necessary  qualifications  were  impossible  to  find  in  large  numbers  at  a  given 
point  in  time.  Altogether,  data  collection  was  underway  during  summer  and 
winter  sessions  of  1979  and  spring  session  of  1980. 

Description  of  Tasks 

The  basic  job  involved  rating  and  coding  information  from  videotaped  se¬ 
lection  interviews  that  were  conducted  at  the  Interviewing  Institute  at  the 
University  of  Houston.  The  particular  videotapes  used  for  this  research  came 
from  a  larger  set  used  by  Howard  and  Daily  (1979).  Care  was  taken  in  selection 
of  videotapes  to  assure  that  interviews  were  of  equal  length  (28-30  minutes) , 
similar  interest  value,  similar  audio  and  visual  quality  and  not  embarrassing 
to  participants.  All  subjects  were  shown  the  same  videotapes,  but  different 
videotapes  were  used  across  the  two  work  sessions.  Subjects  were  aware  that 
applicants  were  applying  for  the  job  of  assistant  manager  in  a  quick  service 
grocery  store  chain. 

To  create  an  enriched  and  an  unenriched  task,  the  five  core  job  dimensions 
were  manipulated  as  follows: 

Autonomy.  The  enriched  task  provided  the  subjects  with  complete  control 
over  any  decision  made  in  the  evaluation  of  the  applicant  and  the  interviewer. 

In  the  unenriched  task,  subjects  were  provided  stop  watches  and  directed  to  time 
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and  record  how  long  the  interviewer  talked. 

Skill  Variety.  The  enriched  task  required  the  subject  to  evaluate  two 
taped  job  interviews  and  make  judgments  on  several  dimensions  regarding  both 
the  applicant  and  the  interviewer.  For  example,  subjects  had  to  evaluate 
the  degree  the  interviewer  actively  listened  to  the  applicant  and  whether 
or  not  the  interviewer  obtained  relevant  material  regarding  the  applicant's 
qualifications.  The  unenriched  task  required  very  little  skill  beyond  being 
able  to  run  a  stopwatch  and  record  time. 

Task  Identity.  The  enriched  task  entailed  a  complete  evaluation  of  each 
interview.  The  unenriched  task  involved  only  a  minor  part  of  the  interview 
and  it  was  stressed  that  at  a  later  date  a  more  complete  evaluation  of  the 
same  interviews  would  be  done  by  other  raters. 

Feedback.  After  each  tape  was  run  in  the  enriched  task  and  subjects  had 
rated  both  the  interviewer  and  the  applicant,  discussion  was  held  among  the 
subjects  on  how  they  rated  the  applicant  and  interviewer.  In  this  way,  sub¬ 
jects  received  some  feedback  as  to  how  they  were  doing  relative  to  their  co¬ 
workers.  In  the  unenriched  task  no  feedback  was  given  concerning  the  correct 
time  that  an  interviewer  talked  during  a  given  tape  nor  were  subjects  allowed 
to  discuss  the  amount  of  time  they  had  recorded  for  a  given  interviewer. 

Task  Significance.  The  enriched  task  was  presented  as  part  of  a  validity 
•study  being  conducted  by  the  Interviewing  Institute.  Subjects  doing  the  en¬ 
riched  task  were  provided  with  a  number  of  reasons  why  such  a  study  is  impor¬ 
tant  and  the  different  people  that  could  be  affected  by  outcomes  of  the  study. 
Also,  they  were  told  that  their  participation  in  the  study  might  help  develop 
a  new  work-study  job  at  the  University  of  Houston.  The  unenriched  task  was 
presented  as  a  "necessary  evil — something  that  needed  to  be  done  to  store  in 
the  Institute's  data  files." 
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In  order  to  insure  that  these  manipulations  had  the  desired  effect ,  a 
pilot  study  was  conducted  (N  =  12)  in  which  the  group  worked  first  on  the 
unenriched  task  and  then  on  the  enriched  task.  Using  dependent  t-tests  on 
comparisons  for  the  five  job  characteristics  as  measured  by  the  JDS  at  Pre 
and  Post,  differences  between  means  for  all  dimensions  except  task  signifi¬ 
cance,  were  significant;  p  <  .01,  and  in  the  expected  direction.  The  mani¬ 
pulation  was  considered  successful. 

Dependent  Measures 

The  short  form  of  the  JDS  was  used  to  measure  perceptions  of  the  task, 
internal  work  motivation  (IWM),  growth  need  strength  (GNS)  and  general  job 
satisfaction  (GS)  (Hackman  and  Oldham,  1975).  Retrospective  Then  ratings 
were  obtained  for  perceptions  of  the  core  job  dimensions  following  recom¬ 
mendations  by  Howard  et  al  (1979).  At  Post -test,  subjects  responded  to  each 
item  twice.  They  first  rated  the  second  session  task  on  a  particular  item 
and  then  were  asked  to  rate  the  first  session  task  on  the  same  item  as  they 
now  perceived  it  to  have  been  prior  to  the  job  change. 

Although  it  was  possible  to  collect  measures  of  work  performance  quality 
on  both  enriched  and  unenriched  tasks,  due  to  the  nature  of  the  job  redesign, 
it  was  not  meaningful  to  compare  differences  in  quality  across  the  two  tasks 
as  a  function  of  differences  in  MPS  scores.  But,  it  would  be  reasonable  to 
assume  that  changes  in  MPS  scores  from  unenriched  to  enriched  tasks  might  pre¬ 
dict  quality  of  performance  on  the  enriched  task.  People  who  perceive  substan¬ 
tial  improvements  as  a  result  of  job  redesign  should  be  most  highly  motivated 
to  do  high  quality  work  on  the  enriched  task.  This  proposed  analysis,  however 
becomes  a  between  subjects  test  of  the  model  and  Is  less  appropriate  than  a 
within  subjects  test. 
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Specifically,  quality  performance  on  the  enriched  task  was  operationalized 
in  a  manner  similar  to  the  use  of  the  distribution  mean  as  an  index  ol  the 
true  score  in  classical  psychometric  theory  (cf.  Lord  &  Novick,  1968).  Each 
subject  provided  an  overall  rating  of  the  effectiveness  of  the  interviewer 
for  each  of  two  videotapes  used  in  the  enriched  condition.  The  mean  rating 
for  each  interviewer  across  subjects  was  computed.  These  values  were  con¬ 
sidered  the  best  estimates  of  the  "true"  ratings.  This  "true"  rating  for  each 
interviewer  was  subtracted  from  each  individual  rating.  The  absolute  values 
of  these  scores  were  computed  and  the  two  scores  were  averaged.  The  result  was 
interpreted  as  an  index  of  error  for  each  subject.  The  larger  the  individual 
deviation  from  the  "true"  rating,  the  lower  the  quality  of  per  f  ormance . 

Design  and  Analysis 

In  order  to  investigate  questions  raised  in  the  introduction,  five  treat¬ 
ment  conditions  were  used.  These  conditions  are  diagramed  in  Table  1. 

=  =s=ss  ==a  =  ss  =  =  =  =s5s  =  =  =  =:5s  =  =  =  s=  =  = 

Insert  Table  1  About  Here 

The  first  four  conditions  presented  the  unenriched  task  first  and  the 
enriched  task  second.  The  fifth  condition  was  one  of  the  many  controls  and 
consisted  of  two  unenriched  task  sessions.  Conditions  three  and  four  deserve 
special  mention.  To  address  the  methodological  question  of  whether  or  not 
asking  for  Then  scores  affects  Post  ratings,  subjects  in  condition  three  com¬ 
pleted  the  Post  questionnaire  and  essentially  believed  that  the  task  was 
over.  They  then  were  presented  with  a  second  brief  questionnaire  and  asked 
to  provide  the  Then  ratings.  Similarly,  to  address  the  methodological  ques- 
t.  ton  of  whether  Then  ratings  affect  Post  ratings,  subjects  in  condition  four 
completed  the  Then  questionnaire  and  once  this  was  handed  to  th<-  experimenter. 


r 
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the  Post  questionnaire  was  administered.  This  procedure  provided  maximum 
data  collection  from  a  limited  number  of  available  subjects.  For  all  other 
subjects,  Post  and  Then  ratings  were  collected  simultaneously  on  the  same 
questionnaire  in  accordance  with  procedures  recommended  by  Howard  et  al  (1979). 

Procedure 

When  subjects  arrived  at  the  workrooms  they  were  read  a  brief  description 
of  the  type  of  task  on  whicn  they  were  going  to  work  and  a  statement  of  the 
purpose  of  the  study.  They  then  provided  biographical  informat  ion  and  respon¬ 
ded  to  the  growth  need  strength  section  of  the  JDS.  Upon  completion  the  ex¬ 
perimenter  told  them  about  the  unenriched  task  and  explained  how  to  record  the 
amount  of  time  an  interviewer  talked  during  an  interview.  A  brief  praet  ice 
period  was  conducted  to  assure  operation  of  the  stop  watches.  Subjects  then 
viewed  and  timed  three  consecutive  videotapes.  After  the  last  tape,  subjects 
completed  the  Pre-questionnaire  and  were  reminded  to  return  for  the  second 
session.  Subjects  spent  a  total  of  two  and  one-half  hours  in  this  session. 

The  second  session  consisted  of  doing  the  enriched  task  and  this  was 
held  within  10  days  of  the  first  session.  Subjects  were  told  that  all  the 
tapes  that  had  to  be  timed  were  finished  and  that  now  their  help  was  needed 
in  determining  the  validity  of  the  Interviewing  Institute.  The  enriched  task 
was  described  and  the  importance  of  doing  the  task  was  emphasized.  Subjects 
then  watched  and  evaluated  two  videotaped  interviews.  After  the  last  tape, 
subjects  completed  the  Post  and  Then  questionnaires.  Subjects  spent  a  total 
of  two  and  one-half  hours  in  this  session.  In  the  control  condition  subjects 
worked  on  the  unenrlcheci  job  for  both  sessions. 


At  conclusion  of  the  second  session,  subjects  were  thoroughly  debriefed 
and  asked  not  to  discuss  the  true  nature  of  the  project  with  students  in  other 
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classes  who  might  be  in  different  treatment  conditions. 

RESULTS 

Methodological  Question  No.  1:  Are  the  different  experimental  gr o  up  s  equ  i  v  a  - 
lent  at  Pre-Test? 

A  one-way  MANOVA  using  Pre-test  scores  on  five  dependent  variables 
yielded  no  statistically  significant  differences  among  the  groups,  Multi¬ 
variate  _F  (15,88.73)  =  .718,  p  =  .76.  The  means,  standard  deviations  and  uni¬ 
variate  F  tests  for  each  variable  are  presented  in  Table  2.  These  findings 

Insert  Table  2  About  Here 

suggest  that  even  without  random  assignment  of  subjects  to  conditio,  s,  the  groups 
are  very  similar  with  respect  to  these  five  important  variables.  An  inspection 
of  standard  deviations  reveals,  however,  that  at  least  with  regard  to  MPS  scores 
Lhere  is  considerable  disagreement  as  to  how  the  task,  is  perceived  within  each 
group.  These  high  standard  deviations  underscore  the  importance  of  testing 
the  Hackman  and  Oldham  model  using  a  within-subj ects  design  as  done  in  this 
study.  It  is  also  important  to  note  that  the  mean  MPS  scores  across  all  four 
groups  are  extremely  low,  the  highest  being  56.54.  The  lowest  MPS  score  re¬ 
ported  by  Hackmand  and  Oldham  (1975)  was  115  for  the  Maintenance-Service  cate- 
go-  y.  The  low  MPS  found  in  this  study  could  be  interpreted  as  proof  that  the 
timing  task  was  indeed  perceived  as  being  unenriched.  Another  explanation  for 
these  results,  however,  might  be  that  the  subjects  were  not  very  involved  in 
tiie  study  and  consequently  put  little  effort  into  rating  the  job  characteristics. 
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Methodological  Question  No.  2:  Does  administering  a  Pre-test  measure  affect 
either  the  Post  or  Then  measure? 

A  t-test  (using  separate  variance  estimates)  was  used  to  compare  the 
means  of  Group  i  and  Group  2  on  the  MPS  variable  for  both  Post  and  Then 
measures.  Differences  between  the  means  for  both  measures  were  non-sign  if i- 
cant:  Post  measure  of  MPS,  t(17.9)  =  1.06,  p  »  .30;  Then  measure  of  MPS, 
t(14.9)  =  1.56,  p  =  .14.  The  means  for  Group  1  and  Group  2  were  respective¬ 
ly,  104.19  and  75.81  for  the  Post  measure  and  31.57  and  12.66  for  the  Then 
measure.  The  standard  deviations  for  Groups  1  and  2  were  70.92  and  48.57 
(Post  measure)  and  39.22  and  12.46  (Then  measure).  The  effect  of  administer¬ 
ing  a  Pre-test  might  be  expected  to  increase  the  Post-test  scores  either 
through  practice  effects  (Campbell  and  Stanley,  1963)  or  through  priming 
(Salancik  and  Pfeffer,  1977).  Although  Post-test  and  Then  measures  were 
higher  in  Group  1,  there  were  no  statistically  significant  differences  be¬ 
tween  the  two  groups.  These  data  provide  some  evidence  that  the  use  of  the 
Pre-test  does  not  automatically  influence  either  the  Post  or  Then  rat ings  of 
MPS. 

Methodological  Question  No,  3;  Does  administering  a  Post-test  affect  t lit1 
Then  rating? 

When  making  a  Post  rating  at  the  same  time  as  the  Then  rating  the  Post 
rating  may  act  as  a  "primer"  by  drawing  attention  to  the  contrast  between 
the  unenriched  and  enriched  tasks.  If  this  were  the  case.  Then  scores  should 
be  noticeably  lower  in  the  group  where  the  Post-test  was  administered  simul¬ 
taneously  with  Then  ratings.  A  t-test  was  used  to  compare  the  means  of  Group 
1  and  Group  4  on  the  Then  MPS  rating.  The  difference  between  the  means  was 
non-significant:  t ( 1 5 . 1 1)  -  .82,  p  ■  .42.  Means  for  Groups  1  and  4  respec- 
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lively  were  31.66  and  21.39  and  standard  deviations  were  39.22  and  15.39. 
Although  the  mean  was  higher  when  Post  scores  were  collected,  it  appears 
that  giving  a  Post-test  measure  at  the  same  time  as  a  Then  measure  has 
little  effect  on  the  Then  MPS  measure. 

Methodological  Question  No.  4;  Does  administering  the  Then  measure-  affect 
t he  Post-test  measure  of  MPS? 

It  is  possible  that  filling  out  the  Then  measure  at  the  same  time  as 
the  Post  measure  may  artificially  inflate  the  Post  measure  either  due  to  un¬ 
foreseen  demand  characteristics  present  during  the  experiment  or  to  hypo¬ 
thesis  guessing.  If  this  were  the  case,  the  group  without  the  Then  measure 
should  have  significantly  lower  Post  KPS  scores  than  the  group  in  which  the 
Then  measure  was  administered  along  with  the  Post  measure.  In  order  to  test 
this  question  a  t-test  was  used  to  compare  Groups  1  and  3  on  the  Post -test 
measure  of  MPS.  There  was  no  significant  difference  between  the  two  means: 
t(.18.99)  =■  .70,  p  =  .49.  The  means  for  Group  1  and  Group  3  were  104.19  and 
85.4  respectively.  The  standard  deviations  for  the  two  groups  were  70.92 
and  51 . 52 . 

The  combined  results  of  these  questions  point  to  two  conclusions.  The 
administration  of  any  one  of  the  types  of  measures  (Pre,  Post  or  Then)  seems 
to  have  little  impac*  on  tne  other  measures,  and  there  seems  to  be  a  consider¬ 
able  range  of  MPS  scores  among  subjects  as  to  the  nature  of  a  particular  task. 

Methodological  Question  No,  5:  Does  knowledge  of  Post-test  responses  at  1 ect 
Then  responses  and  vice  versa? 


Another  methodological  concern  raised  by  using  a  retrospective  Then 
measure  was  whether  order  of  presentation  of  the  Post  and  Then  measures  would 
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have  any  effect.  It  is  possible  that  responding  to  the  Then  question  might 
draw  attention  to  how  "bad"  or  unenriched  the  first  task  was,  and  this  could 
produce  an  artificially  high  Post  score.  On  the  other  hand,  it  is  conceiva¬ 
ble  that  answering  the  Post  measure  on  the  enriched  task  might  cue  the  sub¬ 
jects  to  provide  an  especially  negative  rating  for  the  unenriched  task.  To 
test  these  questions.  Groups  3  and  4  were  compared  on  both  the  Post  and  Then 
MPS  responses  using  separate  t-tests  (pooled  variance  estimate).  Results  were 
as  follows:  Post  measure  of  MPS,  t(16)  -  .91,  p  =  .39;  Then  measure  of  MPS, 
t(26)  =  1.96,  p  =  .07.  Means  and  standard  deviations  for  the  Post  measures 
were  as  follows:  Group  3  X  ■  85.40,  s.d.  =  51.52;  Group  4  X  =  119.06,  s.d.  = 
98.87.  Means  and  standard  deviations  for  the  Then  measures  were  as  follows: 
Group  3  X  =  10.22,  s.d.  -  7.5;  Group  4  X  =  21.39;  s.d.  «*  15.39.  Inspection 
of  these  statistics  suggests  that  responding  to  the  Then  measure  before 
filling  out  the  Post  measure  seems  to  have  little  effect  on  the  Post  measure, 
but  that  responding  to  the  Post  measure  before  filling  out  the  Then  measure 
may  influence  the  Then  measure  by  lowering  the  rating.  This  interpretal ion 
is  proposed  with  two  reservations.  The  comparison  between  the  two  means  lor 
the  Then  measure  was  only  marginally  significant  (p  =  .07).  And,  15  statisti¬ 
cal  tests  were  done  in  this  section  on  methodological  questions  without  con¬ 
trolling  the  alpha  level  across  the  family  of  tests.  It  is  possible  that  the 
marginally  significant  result  in  this  one  comparison  was  a  function  of  the 
large  number  of  tests  performed  on  the  data.  But,  we  also  should  note  that 
the  tests  probably  had  low  power  given  the  sample  size. 

Methodological  Question  No.  6;  Did  the  manipulation  have  an  ef feci  us ing  the 
conventional  Pre/Post  measures? 

Because  the  groups  were  not  significantly  different  at  Pro-test  and  the 
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various  testing  procedures  did  not  seem  to  produce  effects  confounded  with 

the  treatment  effects,  Groups  1,  3,  and  4  were  combined  and  tested  against 

Group  5  on  three  Post-Pre  change  score  variables. 

2 

An  a  priori  Hotelling  T  multivariate  comparison  was  conducted  to  in¬ 
vestigate  differences  between  the  two  groups  on  changes  in  internal  work  moti¬ 
vation  (IWM) ,  general  satisfaction  (GS)  and  motivating  potential  (MPS) .  Re¬ 
margin,!  Uy 

suits  yielded  a/significant  overall  difference  between  the  groups:  Multi¬ 
variate  F(3.34)  =  3.44,  p  =  .08.  The  means,  standard  deviations  and  univariate 
F  tests  for  each  variable  are  presented  in  Table  3.  Inspection  of  the  means 

Insert  Table  3  About  Here 

and  univariate  _F  tests  indicates  that  the  change  from  the  unenriched  job  to  the 
enriched  job  was  successful  in  increasing  the  motivating  potential  score,  in¬ 
ternal  work  motivation  rating,  and  general  satisfaction  rating  of  the  experi¬ 
mental  subjects  (Groups  1,  3,  and  4)  relative  to  the  control  group  subjects 
(Group  5).  These  data  provide  evidence  that  the  manipulation  had  the  desired 
effect  and  that  the  Post-Pre  method  of  measurement  was  able  to  detect  that 
effect.  These  results  support  the  Hackman  and  Oldham  (1976)  model,  which  state 
that  increases  in  MPS  should  lead  to  increases  in  both  work  motivation  and 
general  satisfaction. 

Re  search  Question  No.  1;  Can  the  response  shift  be  shown  to  exist  in  ratings 
of  job  characteristics? 

A  dependent  t-test  was  performed  on  the  Pre  and  Then  measures  of  MPS  for 
Groups  1,  3,  and  4.  The  means  for  Pre  and  Then  respectively,  were  31.53  and 
22.11  and  the  standard  deviations  were  53.20  and  27.30.  The  difference  be¬ 
tween  the  means,  although  in  the  predicted  direction,  was  not  statistically 
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significant:  t(29)  =  .90,  p  =  .37.  Consequently,  a  response  shift  was  not 

present  in  the  rating  of  job  characteristics  with  these  data. 

Research  Question  No.  2:  Does  the  retrospective  Then  rating  provide  a  more 
valid  predictor  of  individual  change? 

In  order  to  answer  this  question,  the  relative  predictive  abilities  of 
five  variables  (change  in  MPS  score  from  Pre  to  Post,  change  in  MPS  from 
Then  to  Post,  growth  need  strength  and  the  interaction  of  the  two  MPS  mea¬ 
sures  and  GNS)  were  investigated  in  a  number  of  hierarchical  regression  equa¬ 
tions  using  the  dependent  variables  of  Post-Pre  change  in  general  satisfac¬ 
tion,  Post-Pre  change  in  internal  work  motivation,  and  Post  performance.  In 
the  analysis  of  the  first  three  dependent  variables  all  groups  were  included 
except  Group  2  because  no  Pre-test  was  given  to  that  group.  Tn  the  analysis 
of  the  Post  performance  measure  only  groups  that  worked  on  the  enriched  task 
and  had  Pre,  Post  and  Then  measures  were  included  in  the  analysis  (Groups  1,  3, 
and  4).  Results  of  these  analyses  are  presented  in  Tables  4,  5,  and  6.  They 

Insert  Tables  4,  5,  and  6  About  Here 

suggest  that  regardless  of  position  in  the  regression  equation  or  the  particu¬ 
lar  dependent  variable  examined,  the  change  in  MPS  scores  (both  Post-Pre  and 
Post-Then  versions)  accounts  for  the  most  variance  in  the  change  in  dependent 
variables.  For  general  satisfaction  and  internal  work  motivation  the  MPS 
change  scores  are  the  only  independent  variables  that  account  for  a  signifi¬ 
cant  proportion  of  the  variance.  These  results  support  the  contention  that 
changes  in  satisfaction  and  internal  work  motivation  should  be  predictable 
from  changes  in  MPS  scores  (Hackman  and  Oldham,  1976),  but,  these  results  do 
not  support  the  use  of  GNS  as  a  moderator  of  individual  reactions  to  job  on- 
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richment  or  the  hypothesis  that  job  enrichment  and  quality  of  job  perfor¬ 
mance  are  positively  related  (Hackman  and  Oldham,  1976).  The  regression 
results  do  indicate  that  both  types  of  MPS  change  scores  (Post-Pre  and  Post- 
Then)  can  predict  changes  in  satisfaction  and  work  motivation.  A  t-test 
for  comparing  differences  between  dependent  correlation  coefficients  (McNamar 
1969,  p.  158)  was  used  to  test  for  significant  predictive  differences  be¬ 
tween  the  two  methods.  There  were  no  significant  differences  between  the 
Post-Pre  method  and  the  Post-Then  method  on  any  of  the  dependent  variables, 
it  seems  that  both  methodswere  equally  valid  in  predicting  individual  change 
in  satisfaction  and  motivation.  This  finding  is  not  unreasonable  given  the 
lack  of  evidence  for  a  response  shift. 

DISCUSSION 

Howard's  work  on  the  use  of  retrospective  Then  scores  in  evaluating  sub¬ 
ject  reactions  to  planned  interventions  (Howard  et  al,  1979)  was  investigated 
as  having  potential  to  provide  more  accurate  tests  of  job  redesign  changes. 
The  Job  Characteristics  Model  (Hackman  and  Oldham,  1976)  was  chosen  for  study 
as  it  currently  is  the  dominant  paradigm  in  the  task  design  area  (Evans, 
Kiggundu  and  House,  1979).  In  addition,  some  field  experiments  of  this  model 
have  produced  unexpected  results  that  could  be  explained  through  use  of  the 
response  shift  concept.  We  believed  a  contribution  would  be  made  to  the  job 
design  literature  if  we  could  demonstrate  the  validity  of  retrospective  Then 
measures  and  their  utility  in  assessing  change. 

Results  of  a  controlled  experimental  simulation  supported  the  validity 
of  the  retrospective  Then  measurement  technique.  Retrospective  ratings  were 
not  related  to  measurement  bias  and  Post-Then  scores  accurately  predicted 


change  in  attitudes. 
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Turning  first  to  the  six  methodological  questions,  the  obtained  re¬ 
sults  suggest  that  the  retrospective  Then  methodology  is  not  highly  sus¬ 
ceptible  to  problems  associated  with  order  effects,  saliency  or  priming. 

Asking  for  Pre  ratings  and/or  Post  ratings  of  job  characteristics  did  not 
significantly  impact  on  Then  ratings  in  this  study.  Similarly,  asking  for 
Then  ratings  did  not  impact  on  Post  ratings.  Pre  ratings  also  had  no  im¬ 
pact  on  Post  ratings  as  typically  assessed  in  past  experimental  studies  of 
job  change. 

These  results  are  important,  especially  as  they  were  produced  in  an 
experimental  simulation  where  demand  characteristics  might  be  strong.  Most 
recently,  Salancik  and  Pfeffer,  (1977)  and  Koch  and  Rhodes  (1979)  raised 
concerns  over  use  of  obtrusive  survey  data  collection  methods  in  tests  of 
planned  change.  Pre-test  data  collection  may  encourage  subjects  to  bring 
attitudes  and  perceptions  into  conformity  with  one  another.  Specific  aspects 
of  the  job  are  made  salient  as  a  result  of  Pre-test  measurement  and  this  may 
impact  on  selective  perceptions,  perceptual  distortion,  and  perceptual  con¬ 
sistency.  Pre-test  measurement  also  may  prime  or  educate  subjects  regarding 
appropriate  or  desired  responses.  Saliency  and  priming  represent  factors  that 
could  effect  Post  or  Then  ratings  of  job  characteristics  independent  of  any 
actual  changes  made  in  the  nature  of  the  job. 

The  present  study  offers  one  point  arguing  against  the  automatic 

biasing  effect  of  these  threats  to  internal  validity.  This,  of  course,  does 
not  imply  that  saliency,  priming  and  order  effects  are  pseudo  problems.  But, 
in  the  absence  of  properly  designed  research  dealing  with  this  question  the 
non-equivalent  control  group  design  is  the  most  common  design  of  field  re¬ 
search  on  change  (Porras  and  Berg,  1978)  —  our  results  would  seem  to  make  a 
contribution.  These  data,  in  combination  with  Howard's  finding  that  retro- 
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spective  Then  ratings  were  not  biased  by  memory  distortion  or  social  de¬ 
sirability,  offer  encouraging  support  for  future  use  of  retrospective 
ratings  following  change. 

Results  pertaining  to  the  two  research  questions  are  somewhat  more 
difficult  to  evaluate.  Using  a  within  subjects  design  that  related  indi¬ 
vidual  change  in  MPS  scores  to  individual  change  in  internal  work  motiva¬ 
tion  and  general  satisfaction,  strong  support  was  found  for  Hackman  and 
Oldham's  (1976)  hypothesized  change  in  dependent  variables.  This  occurred 
regardless  of  whether  perceptions  of  change  were  assessed  with  Post-Pre 
scores  or  Post-Then  scores.  There  was  no  support  for  the  hypothesized  change 
in  quality  of  performance  or  for  the  moderating  effect  of  growth  need  strength. 
This  pattern  of  results  Is  consistent  with  Umstot  et  al  (1976).  The  lack  of 
relationship  between  changes  in  MPS  scores  and  performance  quality  found  in 
our  study  should  be  viewed  with  some  reservation,  however.  The  criterion 
measure  was  essentially  a  between  subjects  index  whereas  the  predictor  was  a 
within  subjects  index.  It  was  not  possible  to  construct  a  meaningful  within 
subjects  measure  of  performance.  Also,  the  construct  vaididity  of  our  error 
index  is  unknovn. 

Although  the  test  of  the  Job  Characteristics  Model  was  essentially  a 
method  check  for  appropriate  evaluation  of  retrospective  Then  ratings,  our 
results  do  have  implications  for  the  job  design  literature.  The  study  was  set 
up  as  a  within  subjects  test  of  the  model.  This  design  more  closely  resembles 
the  dynamics  of  the  hypothesized  relationships.  The  importance  of  such  designs 
was  underscored  by  the  relatively  large  standard  deviations  found  in  MPS  scores. 
In  contrast,  previous  tests  of  the  Job  Characteristics  Model  have  either  failed 
to  manipulate  the  job  or  used  between  subjects  designs.  The  present  study 
also  included  a  control  group  and  checks  on  the  impact  of  saliei.cy  and  priming 
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on  subjective  ratings.  We  are  not  proposing  that  a  controlled  simulation 
lasting  a  total  of  five  hours  with  college  students  as  subjects  be  evaluated 
as  a  superior  method  for  testing  Hackman  and  Oldham's  model.  But,  this 
method  was  appropriate  for  evaluating  the  retrospective  Then  approach  to 
assessing  change.  And,  in  order  to  accomplish  that  goal  we  needed  to  demon¬ 
strate  some  support  for  the  Job  Characteristics  Model.  Post-Then  scores 
were  sensitive  to  changes  in  job  characteristics  and  these  scores  did  pre¬ 
dict  changes  in  general  satisfaction  amd  internal  work  motivation. 

The  utility  of  retrospective  Then  ratings  as  applied  to  tests  of  planned 
job  redesign  is  still  partially  unresolved.  The  technique  survived  all 
hurdles  except  one.  There  was  no  evidence  of  a  response  shift.  MPS  Pro 
ratings  were  not  significantly  different  from  MPS  Then  ratings.  Although 
Then  ratings  were  lower  as  would  be  expected  from  Howard's  research  (Howard 
et  al,  1979),  there  was  no  shift  in  anchor  points  or  frames  of  reference. 

Lack  of  support  for  a  response  shift  would  occur  for  two  reasons.  The  tech¬ 
nique  is  not  sensitive  to  detection  of  response  shifts  or,  in  this  particular 
study,  a  response  shift  did  not  happen.  We  will  address  each  of  these  issues. 

The  work  by  Howard  and  his  colleagues  on  the  response  shift  is  extensive 
(cf.  Terborg  et  al,  1980).  Howard  has  addressed  the  notion  of  subject  recal¬ 
ibration  as  a  problem  when  Pre  and  Post  designs  are  used.  He  has  developed 
a  method  for  detecting  response  shifts,  i.e.,  when  Pre  4  Then,  and  a  method 
for  assessing  change  when  response  shifts  are  found,  i.e.,  Post  -  Then,  He 
does  not  say  that  response  shifts  will  always  result  from  planned  interventions. 
Because  Howard  has  found  considerable  evidence  for  response  shifts,  our  inabilit 
to  detect  a  response  shift  in  this  one  study  would  not  imply  that  the  technique 
is  insensitive  to  subject  recalibration.  The  accumulated  evidence  is  very  sup¬ 
portive  of  the  use  of  retrospective  ratings  to  assess  perceptions  oi  change. 
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But,  we  must  point  out  one  potentially  important  difference  between  this 
study  and  Howard's  research.  All  of  Howard's  investigations  have  focused 
on  a  person's  self  rating  of  skill  or  level  of  functioning.  The  interven¬ 
tion  was  designed  to  change  that  person's  skill  or  behavior.  The  object  of 
the  rating  is  the  person.  In  our  study,  the  object  of  the  rating  was  the 
task.  The  notions  of  anchor  points,  frame  of  reference,  and  recalibration, 
however,  would  seem  to  be  common  to  both  types  of  ratings.  Yet,  it  remains 
possible  that  this  difference  could  produce  insensitivity  to  detection  of 
response  shifts. 

The  second  question  is  whether  the  nature  of  the  task  and  our  changes  of 
that  task  were  capable  of  producing  conditions  where  a  response  shift  might 
be  produced.  Pre  scores  should  not  be  different  from  Then  scores  if  the 
intervention  had  no  impact  on  how  subjects  evaluated  and  rated  concepts  like 
autonomy  and  skill  variety.  Perhaps  one  precondition  for  response  shifts  is 
that  the  rater  have  limited  knowledge  of  the  fulr  range  the  phenomenon  can 
vary.  A  second  precondition  might  be  the  magnitude  of  the  change  caused  by 
the  intervention.  Even  though  our  manipulation  produced  significant  dif¬ 
ferences  in  Pre  and  Post  MPS  scores,  the  nature  of  the  simulated  job  might 
have  limited  the  efficacy  of  the  intervention  to  yield  a  ctiange  in  frame  of 
reference. 

Even  though  we  were  not  able  to  produce  a  response  shift,  we  strongly 
endorse  the  future  use  of  retrospective  ratings  in  studies  of  reactions  to 
change.  A  response  shift  remains  a  viable  rival  hypothesis  to  the  findings 
of  Hackman,  Pearce,  and  Wolfe  (1978),  and  Frank  and  Hackman  (1975).  Inclu¬ 
sion  of  Then  ratings  to  Pre  and  Post  ratings,  which  perhaps  adds  at  most  102 
to  the  time  needed  to  complete  the  survey  instrument,  would  provide  a  valid 
test  of  whether  a  response  shift  had  occurred  and  whether  the  Job  Character- 
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istics  Model  was  supported.  The  retrospective  methodology  also  would  be 
useful  in  studies  examining  the  impact  of  more  extensive  changes  as  a  re¬ 
sult  of  Quality  of  Work  Life  projects  and  organization  development.  Retro¬ 
spective  ratings  easily  could  be  added  to  survey  instruments  commonly  used 
in  these  types  of  studies. 

A  final  comment  about  the  use  of  retrospective  Then  ratings  should  be 
made.  The  retrospective  rating  technique  is  not  a  substitute  for  good  re¬ 
search  methods  and  it  is  not  intended  to  replace  the  collection  and  use  of 
Pre-test  data.  Pre-test  ratings  allow  for  comparison  of  group  equivalency 
prior  to  interventions  and  examination  of  Pre  and  Then  ratings  allows  for 
greater  understanding  of  reactions  to  change  (Terborg,  Howard  and  Maxwell, 
1980).  Knowledge  that  a  response  shift  had  occurred  provides  additional  evi¬ 
dence  that  the  intervention  had  some  effect. 

In  conclusion,  use  of  retrospective  Then  ratings  following  a  planned 
job  change  was  found  to  be  an  internally  valid  method  for  assessing  reactions 
to  change.  The  ability  to  detect  response  shifts  and  to  accurately  measure 
change  when  response  shifts  occur  would  seem  to  be  particularly  useful  in 
future  work  on  job  redesign  because  the  Hackman  and  Oldham  (1976)  model  em¬ 
phasizes  employee  perceptions  of  change. 
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Table  1 

Treatment  Conditions^ 


Group 

Order 

of  Data 

Col lect ion 

Sample 

Size 

Task  Order 

1 

Pre 

Post 

Then 

12 

U-E2 

2 

Post 

Then 

8 

U-E 

3 

Pre 

Then 

(Post ) 

9 

U-E 

U 

Pre 

Post 

( Then ) 

9 

U-E 

5 

Pre 

Post 

Then 

10 

u-u 

'the  measures 
had  completed 

enclosed  with  (  )  were  presented  only  after 
all  other  questionnaires  and  believed  they 

the  subjects 
were  f  in  i  shed 

with  the  task. 

2U  =  Unenriched  task;  E  =  Enriched  task. 
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Table  2 

Means,  Standard  Deviations  and  Univariate  F  Tests: 
Pre-Test  Measures  for  Groups  1,  3,  4  and  3. 


Va r i ab  le 

Group 

1 

Group 

3 

Group 

4 

Group 

5 

F  value* 

r.NS 

X 

6.90 

6.99 

6.42 

7.24 

t 1 .  5  h 

S.D. 

1.02 

2 . 20 

1.32 

0.73 

!  WM 

X 

3.14 

3.41 

4.45 

3.45 

I  .  79 

S.D. 

1.28 

1.80 

0.78 

1  .27 

Sat  i sf act  ion 

X 

3.16 

2.83 

3.44 

2  90 

0.2  5 

S.D. 

1.61 

1 .  76 

1 .68 

1  .  72 

MPS 

X 

20-96 

20.61 

56.54 

30.72 

1 .09 

S.D. 

20.73 

33.51 

86.81 

39.58 

Pe r  f  ormance^ 

X 

0.81 

1.21 

1.14 

1.13 

0.35 

S.D. 

0.67 

0.58 

0.57 

0.55 

for  .ill  of  the  univariate  tests,  p>  .10. 


Performance  measure  for  unenriched  task. was  ope  rat  iona 1 i zed  as  the  average 
f't  fhe  absolute  values  of  deviation  scores  from  the  actual  time  spent  talking 
hv  the  interviewer  in  each  of  three  interviews.  Scores  represent  time  in 
m  i  nut  es  . 
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Table  3 

Means  and  Standard  Deviations  for  Pre  and  Post-test  Variables  and 
Univariate  F  Tests  for  Change  Scores 


Experimental  Control 

Variable  Group  Group  F  Value 


Pre 

Post 

Pre 

Post 

MPS 

X 

32.70 

102.88 

30.72 

31.96 

5.75 

S.D. 

47.04 

73.77 

39.58 

42.64 

IWM 

X 

3.66 

4.58 

3.45 

3.12 

5.87 

S.D. 

1.29 

1.00 

1.27 

0.78 

Sat isf  act  ion 

X 

3.14 

4.81 

2.90 

2.65 

9.88 

S.D. 

1.68 

1.05 

1.72 

1.58 

.05. 


Table  4 


Regression  Results  for  Prediction  of  General  Satisfaction  Change  Scores 


Variables 
in  Step  1 

R2 

Change 

Variables 
in  Step  2 

R2 

Change 

Variables 
in  Step  3 

R2 

Change 

Post-Pre  MPS 

.37 

GNS 

.05 

MPS  x  GNS 

.01 

GNS 

.08 

Post-Pre  MPS 

.34" 

GNS  x  MPS 

.01 

Post-Then  MPS 

.  29 

GNS 

.04 

MPS  x  GNS 

.02 

GNS 

CO 

o 

Post-Then  MPS 

.2  5* 

GNS  x  MPS 

.02 

1  R2  for  the 

R  for  the 

full 

full 

model  using 
model  using 

Post-Pre  MPS  change 
Post-Then  MPS  change 

scores  was  . 

scores  was 

.42  (p<,  .01 )  and 
.35  (p<  .01). 
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Table  5 

Regression  Results  for  Prediction  of  Internal  Work  Motivation  Change  Scores  (N  40)' 


Variables 

R2 

Variables 

R2 

Variables 

R2 

in  Step  1 

Change 

in  Step  2 

Change 

in  Step  3 

Change 

Post-Pre 

MPS 

.35 

GNS 

.02 

MPS  x  GNS 

.03 

GNS 

.03 

Post-Pre  MPS 

■k 

.33 

GNS  x  MPS 

.03 

Post-Then 

MPS 

* 

.27 

GNS 

.01 

MPS  x  GNS 

.03 

GNS 

.03 

Post-Then  MPS 

: k 

.24 

GNS  x  MPS 

.03 

'  R2  for 

the 

full 

model  using 

Post-Pre  MPS  change 

scores  was  .39 

(p<>  .01 )  and 

R  for 

the 

full 

model  using 

Post-Then  MPS  change 

scores  was  .31 

(p<.  .01). 

P<  -05. 
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Table  6 


Regression  Results 

for  Prediction  of 

Post  Test 

Performance  ( N  —  3 0 > 

Variables 
in  Step  1 

R2 

Change 

Variables 
in  Step  2 

R2 

Change 

Variables 
in  Step  3 

R2 

Change 

Post-Pre  MPS 

.03 

GNS 

.05 

MPS  x  GNS 

.00 

GNS 

.04 

Post-Pre  MPS 

.04 

GNS  x  MPS 

.00 

Post-Then  MPS 

CO 

o 

GNS 

c 

00 

MPS  x  GNS 

.01 

GNS 

.04 

Post-Pre  MPS 

.12 

GNS  x  MPS 

.01 

R^  for  the  full  model  using  Post-Pre  MPS  change  scores  was  .08 
R“  for  the  full  model  using  Post-Then  MPS  change  scores  was  .17  NS 
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